The role of learner attributes and affect determining the impact of agent presence

This paper introduces two experimental studies that have examined the efficacy of agent presence in relation to learner attributes and affect. With 132 high-school females, Study I investigated the effects of learners’ prior math attitudes (high vs. low) and prior math selfefficacy (high vs. low) on the changes in their attitudes and self-efficacy after working at a pedagogical-agent-based environment. The results indicated that the females with lowprior-math attitudes significantly increased their math attitudes after working at the environment, whereas the attitudes of females with high-prior-math attitudes were not significantly changed. The same trend was observed for their math self-efficacy. Study II investigated the interaction of learner gender, learner sociability (low-sociable vs. highsociable), and agent presence (present vs. absent) on learners’ math attitudes, math selfefficacy, and learning, with 180 male and female high-school students. The results showed that, for both male and female, low-sociable students had significantly more positive math attitudes after working with an agent than without an agent, whereas high-sociable students had significantly more positive math attitudes after working at the learning environment without an agent than with an agent. The same was true for math self-efficacy. The learners significantly increased their learning regardless of the conditions. The implications of the findings are discussed. Kim, Y. (2009). The role of learner attributes and affect determining the impact of agent presence. International Journal of Learning Technology, 4 (3). 234-249. 3 The Role of Learner Attributes and Affect Determining the Impact of Agent Presence INTRODUCTION Educational theorists and researchers often emphasize the importance of the social context of cognition and its applications to learning and instruction. Learning is a highly social activity. Interactions with teachers, peers, and instructional materials are considered crucial for the cognitive and affective development of learners. Social interaction among participants in learning is seen as the primary source of intellectual development (John-Steiner & Mahn, 2003). This emphasis on social cognition seems to demand reframing the conventional use of computers as cognitive tools (Jonassen, 1995; Lajoie, 2000) and suggests a new metaphor: computers as social cognitive tools (Kim & Baylor, 2006). A pedagogical agent is an animated life-like character (Johnson, Rickel, & Lester, 2000) embedded in an instructional application. The presence of a pedagogical agent is often valued in terms of its potential to build social affective relations with a learner, supported by human/computer interaction research (Nass & Moon, 2000). It sounds natural to propose, then, that the efficacy of agent presence would be influenced by learner variations such as a learner’s affective states and personal attributes. This paper introduces two experimental studies that have examined this proposition. THEORETICAL BACKGROUND Pedagogical Agents In pedagogical-agent-based learning, a learner grasps instructional content while interacting with one or more life-like pedagogical agents. These agents may provide information and/or encouragement, share menial tasks, or collaborate with the learner. What makes pedagogical agents unique from conventional computer-based learning would be their ability to simulate real-world social relations (Bickmore, 2003; Dautenhahn, Bond, Canamero, & Edmonds, 2002). Kim, Y. (2009). The role of learner attributes and affect determining the impact of agent presence. International Journal of Learning Technology, 4 (3). 234-249. 4 In this sense, pedagogical agents may help overcome some constraints of and expand functionalities of conventional technology-based learning environments. Traditionally, computer-based learning (e.g., intelligent tutoring systems) was tailored to meet an individual’s needs, supporting each learner independently when the environments were well designed (Aimeur & Frasson, 1996; J. R. Anderson, Corbett, Koedinger, & Pelletier, 1995; Gertner & VanLehn, 2000; Graesser, VanLehn, Rose, Jordan, & Harter, 2001). However, those learning environments might have limitations in simulating situated social interaction that is regarded as a significant influence on both learning and motivation (Lave & Wenger, 2001; Palinscar & Brown, 1984; Powell, Aeby, & Carpenter-Aeby, 2003; Vygotsky, Cole, John-Steiner, Scribner, & Souberman, 1978; Wertsch, Minick, & Arns, 1984). Pedagogical agents can be designed to support the social affective aspect of learning, playing well-defined instructional roles, following specified social conventions, and even responding to learners with apparent empathy (Hays-Roth & Doyle, 1998). Earlier, Reeves and Nass (1996) concluded, from more than ten years of studies, that people applied the same social rules and expectations to computers as they did to humans in the real world. The simulated social presence of pedagogical agents in a computing environment may provide a learner with a sense of companionship and have the learner perceive the environment as relevant and meaningful (Biswas, Schwartz, & Bransford, 2001). Along the line, it has been desired to equip a pedagogical agent with personas. To build social relations, a pedagogical agent should have human-like personas to be natural and believable (Bates, 1992; Lester, Voerman, Towns, & Callaway, 1999; Ortony, 2002). A pedagogical agent is frequently designed to represent a human instructional role, such as expert (Johnson et al., 2000), tutor (Graesser, Moreno, & Marineau, 2003), mentor (Baylor & Kim, 2005), and virtual peer or learning companion (Chan & Baskin, 1990; Dillenbourg & Self, 1992; Goodman, Soller, Linton, & Gaimari, 1998; Hietala & Niemirepo, 1998; Kim, Y. (2009). The role of learner attributes and affect determining the impact of agent presence. International Journal of Learning Technology, 4 (3). 234-249. 5 Kim, 2003; Uresti, 2000). A pedagogical agent designed as a virtual peer adopts a peer metaphor, where the agent who acts as a co-learning peer learns with the learner. The positive impact of pedagogical agents on cognitive and/or motivational outcomes has been supported by some empirical studies. Atkinson (2002) reported that students who received worked-example instruction from animated pedogogical agents reported lower levels of perceived difficulty than did students in the control group who received textual information without the agents and also outperformed the counterparts in both nearand far-transfer tests. Moreno and colleagues showed that students with pedagogical agents produced significantly more correct solutions on difficult transfer problems and rated their interests in the material significantly greater than students without pedagogical agents (Moreno, Mayer, Spires, & Lester, 2001). These authors concluded that students might build a positive personal relationship that promoted their interest and learning. Kim and colleagues found that male and female college students showed significanlty greater interest and self-efficacy in the task after working with an agent who expressed empathy than did students who worked with a non-empathetic agent (Kim, Baylor, & Shen, 2007). Ryokai and colleagues showed that children who played with the pedagogical agent Sam designed as a virtual peer listened to Sam’s stories carefully and mimicked Sam’s linguistic styles (Ryokai, Vaucelle, & Cassell, 2003). In her review study, Gulz (2004) summarizes the benefits of pedagogical agents: 1) increased motivation, 2) increased sense of ease and comfort in a learning environment, 3) stimulation of essential learning behavior, 4) increased smoothness of information and communication processes, 5) the fulfillment of a need for personal relationships in learning, and 6) gains in terms of memory, understanding and problem solving. Difference in Learner Characteristics Learners differ in their cognition and affect when learning (Ackerman, Kyllonen, & Roberts, 1999). The concept of individual difference has interested educational researchers Kim, Y. (2009). The role of learner attributes and affect determining the impact of agent presence. International Journal of Learning Technology, 4 (3). 234-249. 6 over time since the late 50s and early 60s. Cronbach (1957) suggested that there existed the interaction between aptitude and treatment (ATI), meaning that two treatments might produce different possible payoff functions given learners with varying aptitudes. His idea led to a surge of empirical studies on aptitude-treatment interaction, the results of which have supported the individuals’ varying reactions to the same treatment, which influenced the effectiveness of the treatment. More recently, aptitude has been extended to include other learner characteristics such as prior knowledge, cognitive styles, learning preferences, task-related attitudes, and so on (Hyona, 2002; Larsen & Diener, 1987; Nilsson & Mayer, 2002; Triantafillou, Pomportsis, Demetriadis, & Georgiadou, 2004). The concept of individual difference fundamentally posits that different types of learners might benefit from different instructional approaches. In a Snow & Lohman’s (1984) study, the provision of structured guidance was beneficial to students with low ability more than to their high-ability counterparts. After receiving structured guidance, low-ability students became more positive about their ability and about the learning experiences. Arroyo and colleagues found that concrete hints were more effective for lowability students, whereas highly symbolic hints were more effective for high-ability students (Arroyo, Beck, Woolf, Beal, & Schultz, 2000). Sonnenwald and Li (2003) found that students with a competitive learning style perceived a collaborative learning system more positively, whereas students with an individualistic style perceived the system more negatively. In pedagogical-agent-based learning, Hietala and Niemirepo (1998) showed that the level of learner academic competency determined a learner’s choice of an agent as the collaborating partner. In classrooms, even experienced teachers might be limited in tailoring instruction for each student, due to the lack of resources. Computer-based tutoring may afford some degree of flexibility in that regard. Adaptive e-learning is characterized by identifying and accommodating individual variations among learners. Russell (1997) states that “individual Kim, Y. (2009). The role of learner attributes and affect determining the impact of agent presence. International Journal of Learning Technology, 4 (3). 234-249. 7 differences in learning styles dictate that technology will facilitate learning for some, but will probably inhibit learning for others, while the remainder experience no significant difference (p. 44).” Similarly, the effectiveness of pedagogical agents should not be exceptional and should be influenced by individual differences. The designers of educational systems, therefore, need to be aware of individual differences that would affect motivation and learning and endeavor to address learners’ different cognitive and affective styles and preferences (Hills, 2003). Conventionally, tutoring systems have addressed individual difference to some degree through individualized guidance, but their focus remained mainly on learner cognition. Individual difference, however, can be characterized in multiple dimensions, especially including learner affective characteristics like attitudes toward the task, selfefficacy beliefs in the task, personalities, or emotions (Gagne, Wager, & Rojas, 1981; Jonassen & Grabowski, 1993; Merrill, 2002). In particular, such affective characteristics as task-specific attitudes (Mclnerney & Van Etten, 2002; Shaw & Marlow, 1999) and selfefficacy beliefs (Badura, 1997; Britner & Pajares, 2001) are considered influential for learner engagement and successful learning experience. Nonetheless, not many tutoring systems have paid attention to learner differences in those affective characteristics. As mentioned earlier, the role of life-like pedagogical agents in instructional systems is valued in terms of buiding social relations with a learner through affective interchanges (Bickmore, 2003; Dautenhahn et al., 2002). It would be meaningful to examine the impact of the social presence of a pedagogical agent in relation to learner attributes and affect. This paper introduces two experiments that have examined how different levels of learner attributes and affect would interact with the presence of a pedagogical agent to influence learners’ affect and cognition in learning. Study 1 focused on the differing levels of task-specific attitudes and self-efficacy to examine their impact on the changes in learner attitude and self-efficacy after learners’ working at a pedagogical-agent-based learning Kim, Y. (2009). The role of learner attributes and affect determining the impact of agent presence. International Journal of Learning Technology, 4 (3). 234-249. 8 environment. Study 2 focused on learner gender and learner sociability to examine their interaction with agent presence on learner attitudes, self-efficacy, and learning. Detailed descriptions follow. STUDY 1 Females’ lack of interest in learning STEM (science, technology, engineering, and math) has concerned educators in the USA. In an effort to address this concern, a pedagogical-agent-based learning environment (called MathGirls) was created to investigate the efficacy of a pedagogical agent on persuading high-school females to increase their positive attitudes toward and self-efficacy beliefs in learning math. In the framework of individual difference, the author conjectured that the females’ strong or weak task-specific attitudes and self-efficacy might result in differential impacts of the pedagogical agent. Hence, this experiment examined how the levels of prior math attitudes and prior math self-efficacy would influence the efficacy of agent persuasion at MathGirls. There were two research questions: 1) Will the levels (high vs. low) of learner prior attitudes influence their attitudes changes after working with a pedagogical agent? 2) Will the levels (high vs. low) of learner prior self-efficacy influence their self-efficacy changes after working with a pedagogical agent? Participants Participants were 132 female 9 graders taking required introductory algebra in two high schools located in a mountain-west state of the USA. The ethnic compositions of the participants as self-reported were Caucasian (58.3%), Hispanic (22.8%), African-American (3.9%), Asian (3.3%), and others (11.7%). The average age was 15.51 (SD = 1.14). Intervention The intervention MathGirls was a pedagogical-agent-based learning environment delivered via the web. In Mathgirls, the high-school females practiced algebra problemsolving individually for themselves after taking conventional lessons from their teachers. Kim, Y. (2009). The role of learner attributes and affect determining the impact of agent presence. International Journal of Learning Technology, 4 (3). 234-249. 9 The primary role of a pedagogical agent was to proactively pursuade the females to increase their positive attitudes and self-efficacy in learning math while the agent guided the students through the learning tasks. Figure 1 presents a screenshot of the learning environment. Curriculum “Fundamentals in Algebra” was the curriculum. Following the Principles and Standards of the National Council of the Teachers of Mathematics (NCTM, 2000), the curriculum content dealt with fundamentals in two areas of introductory algebra, each area providing one-class-hour lesson. Lesson I covered combining like terms and the applications of distributive property; Lesson II covered graphing linear equations. Each lesson took one-class period and included four to five subsections that consisted of Reviews and Problem-Solving Practice. Agent and message design The agent was developed using the 3D agent design tool Poser 6 (http://www.efrontier.com). Given that human voice carries affective information beyond the message, the agent voice was pre-recorded by a voice actor. The agent image and voice were integrated within Mimic Pro for lip synchronization. Facial expressions, blinking, and head movements were added to make the agent look natural. Then the 3D animated agent was rendered in Poser to produce AVI files, which were later batch-compressed to be cast via the web. Agent scripts included three types: informational, motivational, and persuasive. The informational message was content-related, including reviews -the brief overviews of what the students had learned from their teachers -and feedback on their performances. When a student made a mistake, the agent provided error-specific explanations to guide her to the right problem-solving path, which helped construct knowledge step by step. The motivational message was words of praise or encouragement. When a student had the Kim, Y. (2009). The role of learner attributes and affect determining the impact of agent presence. International Journal of Learning Technology, 4 (3). 234-249. 10 correct answer, the agent said “Good job” or “Great, I’m proud of you”; when the student had a wrong answer, the agent said “Everybody makes mistakes” or “You’re getting there. One more thing you need to consider is...” The persuasive message was statements about the benefits or advantages of learning math and pursuing careers in STEM. At the beginning of each section, the agent proactively presented the persuasive message, without a learner’s request, to positively influence the females’ attitudes toward and self-efficacy in learning math. Variables and Measures Math attitudes Fishbein and Ajzen (1975) defined an attitude as “ a learned predisposition to respond in a consistently favorable or unfavorable manner with respect to a given object (p. 6). In this study, math attitudes referred to the degree of learners’ favorableness toward learning math (L. W. Anderson & Bourke, 2000). A questionnaire of 10 items was derived from the Mathematics Attitude Survey (Ethington & Wolfe, 1988) and Attitudes Toward Mathematics Inventory (Tapia & Marsh, 2004), with the items scaled from 1 (Strongly disagree) to 7 (Strongly agree): 1) I like math, 2) I enjoy learning math in class, 3) I want to take another math course, 4) I would like to participate or I do participate in extra math activities after school, 5) I think math is an important subject for me to study, 6) I think math is useful in everyday life, 7) I think only smart students can do math, 8) I hate math, 9) Doing math assignments always makes me nervous, and 10) I would not take a math class if I had a choice. Learners’ math attitudes were measured before and after the intervention. The mean scores of the items were calculated. Given the pretest math attitudes, the learners were divided into three levels -of high, mid, and low -by their mean scores. For statistical contrast, a total of 75 females in the highand low-attitudes groups were included in the analysis. Item reliability evaluated with Coefficient α was .87 in the pretest and .79 in